Field of the Invention
The present invention relates to an image stabilization apparatus and an image stabilization method.
Description of the Related Art
Interchangeable lenses and image capturing apparatuses comprising an image stabilization apparatus that detects a camera shake and moves a lens or an image sensor that can move to correct an image shake due to the detected camera shake are known. An image stabilization function of such a method is known as optical image stabilization. Also, in recent years, there are image stabilization functions that, for a moving image, output by extracting a frame image whose position is shifted in a direction that cancels a camera shake, and this is used in small-scale/light weight image capturing apparatuses, and mobile telephones with image capturing apparatuses. Image stabilization of such a method is known as electrical image stabilization.
As a method of detecting a shake, an angular velocity sensor (gyro sensor) is typical, and the shake is corrected by moving a lens or an image sensor in a direction that cancels the shake based on the detected angular velocity. In Japanese Patent Laid-Open No. 2005-43780, it is disclosed that prediction is conducted by feeding back in real time an offset of an angular velocity sensor calculated from a motion vector into an image stabilization drive amount to modify the angular velocity sensor offset.
Also, in recent years, in accordance with the acceleration of image capturing apparatus frame rates and improvements in image processing, techniques that detect a shake by analyzing shake in images between frames to obtain a motion vector are known.
Meanwhile, as advanced imaging methods there is a panning shot, and there is macro imaging where the subject is at a short distance. Macro imaging is when the subject is at a short distance, and in such cases, since a translational shake component of the image capturing apparatus that is in addition to a normal angular shake cannot be ignored, there is an increased need for image stabilization since camera shake is larger than in ordinary imaging. Consequently there are techniques for raising the precision of a camera shake correction amount by detecting the translational shake using an acceleration sensor, and calculating the translational shake component separately from the angular shake. Also, there are techniques for improving an image stabilization effect at a time of macro imaging in cases where no acceleration sensor is comprised by predicting an acceleration component of the image capturing apparatus from a position signal and a drive amount of the image stabilization apparatus, and calculating the translational shake component from the predicted acceleration.
A panning shot is imaging while tilting or panning the image capturing apparatus in accordance with a subject that moves. In such an imaging method, the subject is accentuated and the subject is caused to have motion by imaging by intentionally causing the background of the subject that moves to flow past without causing the subject that is moving to shake. Such an imaging method requires sophisticated technology and so there are techniques that assist by correcting image blur by driving an image stabilization apparatus.
In Japanese Patent Laid-Open No. 2014-211531, in a method of assisting a panning shot, it is disclosed that a panning amount is calculated from a motion vector and an angular velocity detection signal of an angular velocity sensor, and a difference between the angular velocity detection signal and the panning amount is used to execute image stabilization. By this, an adverse effect due to image stabilization in the panning shot assistance function is reduced.
Also, a technique in which, in shake detection by a motion vector, an image is divided into small blocks, a local motion vector is calculated for each block, and then an overall global motion vector is calculated from the plurality of local motion vectors is disclosed in Japanese Patent Laid-Open No. 2008-192060. By this, it is possible to remove elements other than camera shake due to erroneous calculation of a motion vector due to image noise or blurriness due to movement of the object.
However, in image stabilization apparatuses that use an angular velocity sensor, it was not possible to sufficiently raise the performance of image stabilization due to the following two reasons.
The first is offset calculation error in the angular velocity sensor. Because an angular velocity sensor includes sensor specific offset noise and low frequency band noise due to temperature fluctuation, it is not possible to apply a complete integral in integration processing at a time of calculating an image stabilization value. In performing a pseudo integral in a low-pass filter (LPF) for integration processing, it is necessary to sacrifice ultra-low-frequency component performance.
The second is angular velocity sensor sensitivity variation. Various adjustments are made on the image capturing apparatus at the time of shipment in order to reduce sensitivity variation, but an 100% image stabilization effect cannot necessarily be achieved since an adjustment error component necessarily remains.
Also, costs increase due to incorporation of an acceleration sensor in the case where an image stabilization effect at a time of macro imaging is improved by using an acceleration sensor. Meanwhile, in cases where the acceleration on the image capturing apparatus is predicted from the drive amount and the position signal of the image stabilization apparatus, there is a problem in that estimation accuracy is bad if the shake is a minute shake.
Also, in Japanese Patent Laid-Open No. 2005-43780, since the offset of the angular velocity sensor is updated in real time by feedback control, there are cases in which the anti-shake performance deteriorates in the middle of prediction (while waiting for a still image). Though it depends on the design of the feedback gain of the feedback control, the problem that deterioration of anti-shake performance remains when the feedback gain is made to be larger, even though convergence of the prediction is faster. Also, when the feedback gain is made to be smaller, though the extent of the deterioration of anti-shake performance during prediction becomes smaller, the speed of convergence becomes slower.
Also, in Japanese Patent Laid-Open No. 2014-211531, while calculation of the panning amount and panning determination are performed from output of the angular velocity sensor and a motion vector differences, sensitivity variation and an offset component which is angular velocity sensor specific noise have not been considered. For this reason, there are cases in which an error in the calculation result remains, and sufficient panning shot determination precision and assistance precision have not been achieved.